Library Integration for DIS - Preparing the Frames
 | This page page is under construction. |
Preparing Input and Output Images
Before a frame can be processed by the DigitalStabilizer, it must be converted into an RVS image object. The stabilizer expects both input and output images to use the RGBA format and the allocator corresponding to the selected backend.
HostAllocator (OpenCV Backend)
As an example, when capturing frames through OpenCV, the frames are received as BGR images. Therefore, the input frame must first be converted to RGBA and then wrapped into an RVS image object. The conversion and wrapping are performed by the following reference helper:
static std::shared_ptr<HostImageType>
WrapBgrAsRvsRgba(const cv::Mat &bgr,
uint64_t timestamp_us,
cv::Mat *rgba_scratch)
{
cv::cvtColor(
bgr,
*rgba_scratch,
cv::COLOR_BGR2RGBA);
uint8_t *planes[] = {rgba_scratch->data};
return std::make_shared<HostImageType>(
bgr.cols,
bgr.rows,
stride,
planes,
timestamp_us,
nullptr);
}
The helper is typically used as follows:
cv::Mat input_frame; // OpenCV BGR frame cv::Mat rgba_scratch; // Temporary RGBA buffer uint64_t timestamp_us; // Frame timestamp auto rvs_frame = WrapBgrAsRvsRgba( input_frame, timestamp_us, &rgba_scratch);
The output frame preparation is simpler because the output buffer is already allocated as an RGBA image. The output wrapper only adapts the OpenCV buffer to the RVS image abstraction:
static std::shared_ptr<HostImageType>
WrapRgbaAsRvsImage(cv::Mat *rgba,
uint64_t timestamp_us)
{
uint8_t *planes[] = {rgba->data};
return std::make_shared<HostImageType>(
rgba->cols,
rgba->rows,
stride,
planes,
timestamp_us,
nullptr);
}
The output wrapper is typically used as follows:
cv::Mat output_rgba; // RGBA output buffer auto rvs_out = WrapRgbaAsRvsImage( &output_rgba, timestamp_us);
Once the images have been prepared, they can be passed to the stabilizer:
stabilizer.Apply<HostAllocatorType>(
rvs_out,
rvs_frame);
CudaAllocator (CUDA Backend)
For the CUDA backend, the image objects are created once and reused throughout the lifetime of the application.
The input image is created using a CUDA allocator and a pinned RGBA buffer:
uint64_t timestamp_us; rvs_frame_cuda = std::make_shared<CudaImageType>( width, height, input_stride, &d_upload_rgba_pinned, timestamp_us, cuda_allocator);
Likewise, the output image is created using a second CUDA-backed RGBA buffer:
uint64_t timestamp_us; rvs_out_cuda = std::make_shared<CudaImageType>( width, height, output_stride, &d_download_rgba_pinned, timestamp_us, cuda_allocator);
Unlike the HostAllocator path, where new image wrappers are created for each frame and the timestamp is provided during construction, the CUDA images are persistent and reused throughout the lifetime of the application. Therefore, before each call to the stabilizer, the current frame timestamp must be updated explicitly:
rvs_frame_cuda->SetTimestamp(timestamp_us); rvs_out_cuda->SetTimestamp(timestamp_us);
Once the timestamps have been updated, the images can be passed directly to the stabilizer:
stabilizer.Apply<CudaAllocatorType>(
rvs_out_cuda,
rvs_frame_cuda);
